1. Field of the Invention
The present invention relates to a composition technique of a real image and computer graphics (CG) and, more particularly, to an image processing apparatus and method, and a control program, which are suitably used in shooting of movies and television programs, their rehearsals, plays, games, karaoke, and the like.
2. Description of the Related Art
Conventionally, a technique for compositing a picture taken in a studio and a separately prepared CG picture or a picture taken at another location, and recording or broadcasting that composite video as a video is known. In this technique, shooting in the studio is done using a blue or green wall called a blueback as a background, and a composite picture is generated by a method called chromakey for compositing another picture on the blueback portion.
As applied systems of the technique for compositing a real picture and CG picture, for example, Japanese Patent Laid-Open Nos. 11-309269 and 11-88913 have been proposed. In these systems, a real picture is used as a background, and a CG picture or the like is superimposed on that picture to attain a composition process.
Furthermore, as the use pattern of pictures taken in this way, experiments of interactive television systems using the Internet have been extensively made.
However, in the aforementioned method using a blueback, since the performer cannot see a picture to be composited, his or her action may become unnatural or the degree of freedom in action may be reduced.
In the method of Japanese Patent Laid-Open Nos. 11-309269 and 11-88913, when the relationship between a real picture and a picture to be composited is fixed, the positional deviation between the pictures is negligible. However, when a performer, camera, virtual object, and the like move largely and intricately, it is difficult to obtain an accurate composite picture free from any positional deviation.
On the other hand, in recent years, upon development of head-mounted displays, wearable computers, and the like, a performer can act while observing a composite picture in real time. However, practical user services using such devices have not been proposed yet.
In interactive television experiments conducted so far, viewer participation in terms of camerawork and scenario development have been examined. However, in such experiments, performers cannot directly see virtual characters that serve as viewers. For this reason, interactions between the viewers and performers are limited considerably.
In conventional television broadcast, since advertisements are broadcasted in the intervals of a television program, they may be ignored by viewers who are interested in only the television program, and may be cut by a video deck with a CM cut function, or another channel may be selected.
In order to avoid viewers ignoring television advertisements, Japanese Patent Laid-Open No. 10-290443 embeds CM information in contents. However, this method exploits video on-demand, and cannot be used in prevalent television systems.
On the other hand, advertisements such as advertisement comments on the wall of a stadium during a professional baseball live program may appear in a television program, and these advertisements are never cut since they are broadcasted during the program. However, such advertisements cannot be changed during broadcast, and advertisements that appeal more cannot be made.
Also, in interactive television experiments conducted so far, the viewer can arbitrarily manipulate camerawork or camera positions. However, such viewpoints are objective camera viewpoints of a third party, and it is impossible to see a picture from the subjective viewpoint of a performer such as a main cast or the like.
Currently, the viewer can simultaneously watch pictures of a plurality of channels such as the Internet, satellite broadcast, ground wave broadcast, and the like, and a demand has arisen for multi-channel broadcasting pictures at camera viewpoints and performer viewpoints.
The present invention has been made in consideration of the above prior arts, and has as its object to provide a high-quality composite picture of real and virtual pictures to viewers and the like.